Section: Partnerships and Cooperations

European Initiatives

Numerical methods for Markov decision processes (2013-2015)

This project is funded by the Gobierno de Espana, Derccion Genral de Investigacion Cinetifica y Tecnica (reference number: MTM2012-31393) for three years to support the scientific collaboration between Tomas Prieto-Rumeau and François Dufour. This research project is concerned with numerical methods for Markov decision processes (MDPs). Namely, we are interested in approximating numerically the optimal value function and the optimal controls for different classes of constrained and unconstrained MDPs. Our methods are based on combining the linear programming formulation of an MDP with a discretization procedure —referred to as quantization— of a probability distribution, underlying the random transitions of the dynamic system. We are concerned with optimality criteria such as the total expected cost criterion (for finite horizon problems) and, on the other hand, the total expected discounted cost and the average cost optimality criteria (for infinite horizon problems).